Decisions with Big Data

ABSTRACT

This invention presents a framework for applying artificial intelligence to aid with product design, mission or retail planning. The invention outlines a novel approach for applying predictive analytics to the training of a system model for product design, assimilates the definition of meta-data for design containers to that of labels for books in a library, and represents customers, requirements, components and assemblies in the form of database objects with relational dependence. Design information can be harvested, for the purpose of improving decision fidelity for new designs, by providing such database representation of the design content. Further, a retrieval model, that operates on the archived design containers, and yields results that are likely to satisfy user queries, is presented. This model, which is based on latent semantic analysis, predicts the degree of relevance between accessible design information and a query, and presents the most relevant previous design information to the user.

CROSS REFERENCE TO RELATED APPLICATIONS

U.S. Pat. No. 9,923,949 B2, issued on Mar. 20, 2018.

Provisional patent application No. 62/345,760, filed on Jun. 4, 2016.

Utility patent application Ser. No. 15/613,183, filed on Jun. 3, 2017.

Provisional patent application No. 62/583,377, filed on Nov. 8, 2017.

ACKNOWLEDGEMENT OF FEDERAL FUNDING

This utility patent is the result of research conducted under support ofNational Science Foundation Awards 1,447,395 and 1,632,408.

BACKGROUND OF THE INVENTION 1. Technical Field Description

This invention presents a high-level framework for applying artificialintelligence (predictive analytics) to numerous areas of engineeringproduct design, to mission or to retail planning. We also present aquerying engine, utilizing an information retrieval framework, capableof providing teams (workforces) with information similar to the onesharvested in a database, for the purpose of improving efficiency andefficacy. Just as one example involving engineering focus, theinformation provided can consist of design ideas for concept design. Asanother example, the querying engine can, when combined with userinteractions, help with assessment of engineering requirements that aremore difficult to assess automatically.

There presently is significant interest in big data analytics,especially within the automotive industry. Large amounts of data arecollected from fleets of vehicles. The data is being uploaded to cloudsystems, where it is analyzed using big data and machine learningalgorithms. Then, information of interest can be communicated to driversfor system feedback. In addition, some streaming data can be madeavailable to automotive vendors for efficiency and maintenancemonitoring, or used internally by an original equipment manufacturer forpost-mortem failure analysis. Large companies in the automotive industryanalyze the data collected, in search for insights, that can help themprovide better value to their customers, and help them make betterbusiness decisions.

Similarly, CAD design tools generate lots of data. There is interest inharvesting information from past designs, even designs that eventuallyhave not received selection for implementation, but may fulfill thedesign requirements, for the purpose of providing predictions aimed ataiding future design projects. This begs the question: How to optimallymake use of data from past design projects to help with future designs?

Further, as manufacturing processes become more complex, and the dataproduced by different steps in the manufacturing process becomes moreabundant, it has become possible to merge traditional metrology andanalysis techniques with big data concepts.

Another related application involves mission planning. The military hastraditionally relied on information (data) for important decisions.There presently is interest in combining the power of data analyticswith the management and execution of missions, to better arm leaderswith the information they need, when they need it. The data analyticsmay provide insights on patterns in adversarial nations' behaviors tohelp predict when missile launches will occur or when forces are beingmobilized. The analytics can also help identify new transit routes andprepositioning of provisions, to reduce resupply times, and improveresponses to contingencies. And when a situation takes an inevitable orunexpected turn, mission analytics can help leaders respond quickly, andreallocate assets to ensure the force remains ready and optimallyplaced. The purpose of the data analytics is to empower, but notoverwhelm.

Yet, another related application involves retail planning and supplychain management. Large shoe and apparel vendors, such as Nike, havebeen exploring ways to harvest analytics for the purpose of anticipatingcustomers' preferences, customizing (personalizing) and simplifying thecustomers' shopping experience, in particular the digital shoppingexperience.

With utilization of big data on the rise, such as within the automotiveor retail industries, applications to the process of product design ormission planning have been aspiring, but still relatively limited.

2. Description of Prior Art

2.1 Artificial Intelligence and Big Data Analysis in Context with theEngineering Design Process

1. Initial Work

In (Brown 2005), AI was used to improve the way that agents (people ormachines) design things (i.e., to design process improvements). In thisinvention, a framework is presented, which relies on archival of designinformation into properly structured databases, on informationretrieval, and semantic analysis. The framework presented in (Brown2005) is more in line with methods employed by search engines, such asthe one by Google, or by intelligent personal assistants, such as AmazonAlexa (Alexa 2017), Google Assistant (GoogleAssistant 2017) or GoogleHome (Home 2017).

2. Machine Learning for Recommending Design Methods

(Fuge 2014) presents algorithms, that automatically learn patterns indesign methods used, by computationally analyzing design case studies.By using case studies as training data, the algorithms can recommend amethod, or a set of methods, that are most suitable for a given case.

3. Database Structures Harvesting Design Repositories Suitable for BigData Analytics

In (SteingrimssonYi 2017), database structures harvesting designrepositories suitable for big data analytics were presented. Thisinvention provides further context to, and dependencies between, thedatabase structures from (SteingrimssonYi 2017). These database entitiesare presented primarily in the context of mechanical product design andtheir functional origin emphasized.

In addition to (SteingrimssonYi 2017), the big data framework is largelymotivated by Dr. Yi's and Dr. Steingrimsson's previous work from (Yi2009) and (Steingrimsson 2014).

4. Fast Searches of Large Design Databases and Ontology-Based DesignInformation Extraction

The most relevant prior art likely involves (Ramani 2013), (Iyer 2003),(KuiyangLou 2003), (LiRamani 2007) and (SinhaBai 2016).

In (Ramani 2013), (Iyer 2003) and (KuiyangLou 2003), techniques areprovided for searching on three dimensional (3D) objects across large,distributed repositories of 3D models. 3D shapes are created for inputto a search system; optionally user-defined similarity criterion isused, and search results are interactively navigated and feedbackreceived for modifying the accuracy of the search results. Search inputcan also be given by picking 3D models from a cluster map or byproviding orthographic views for the 3D model. Feedback can be given bya searcher as to which models are similar and which are not. Varioustechniques adjust the search results, according to feedback given by thesearcher, and present the adjusted results to the searcher.

(LiRamani 2007) proposes to use shallow natural language processing anddomain-specific design ontology, to automatically construct a structuredand semantic-based representation from unstructured design documents,for design information retrieval. Design concepts and relationships ofthe representation are recognized from the document, based on thelinguistic patterns identified. The recognized concepts andrelationships are joined to form a concept graph. The integration ofthese concept graphs builds an application-specific design ontology,which can be seen as structured representation of content of a corporatedocument repository, as well as an automatically populated knowledgebased from previous designs.

In (SinhaBai 2016), the authors adopt an approach for converting a 3Dshape into a ‘geometry image’, so that standard convolutional neuralnetwork can directly be used to learn 3D shapes. The authorsqualitatively and quantitatively validate that creating geometry imagesusing authalic parametrization on a spherical domain is suitable forrobust learning of 3D shape surfaces. This spherically parameterizedshape is then projected and cut to convert the original 3D shape into aflat and regular geometry image. The authors propose a way to implicitlylearn the topology and structure of 3D shapes using geometry imagesencoded with suitable features. The authors then show the efficacy oftheir approach for learning 3D shape surfaces for classification andretrieval tasks on non-rigid and rigid shape datasets.

Further examples of prior art are referenced in (Ramani 2013), (Iyer2003), (KuiyangLou 2003), (LiRamani 2007) and (SinhaBai 2016).

This invention differs from (Ramani 2013), (Iyer 2003), (KuiyangLou2003), (LiRamani 2007) and (SinhaBai 2016) in the sense that thisinvention analyzes designs at the project (requirement) level. The priorart by Ramani et. al. involves analysis of designs at the solid model(the geometric modeling) level.

5. Application of Artificial Intelligence Towards Faster Design andManufacturing

(BethO'Leary 2018) reports on Dr. Kim and much of the faculty at MIT'sDepartment of Mechanical Engineering creating new software that connectswith hardware to create intelligent devices. They are attempting tobuild an actual physical neural network on a letter paper size. They arebuilding artificial neurons and synapses on a small-scale water. Theyare also looking into AI as a way to improve quality assurance inmanufacturing, and to design more effectively. They refer to it ashybrid intelligence for design. The goal is to enable effectivecollaboration between intelligent tools and human designers.

2.2 Artificial Intelligence or Big Data Analysis in the AutomotiveIndustry

Autonomous driving has been the subject of intense research and muchpublicity, in part due to fatal crashes in California and Arizona(JackStewart 2018), (NathanBomey 2018). Areas of focus have includedmanagement of the data collected, quality of the data, traceability,security, privacy, authentication, telematics, and ultimately ownershipand monetization of the data.

Companies like Daimler Trucks North America operate, and collect datafrom, a fleet of test vehicles, apply AI to identify anomalies in thedata, and analyze. The analysis can include fusion of data fromdifferent types of sensors (cameras, radars and LIDARs), or inertialmeasurement units, real-time localization of objects in the scene, aswell as identification of driver behavior. The goal is to apply big dataanalytics to the large amounts of data collected, for purpose ofidentifying how to offer better value to the customers and make betterbusiness decisions.

2.3 Artificial Intelligence and Big Data Analysis in EngineeringEducation

Dr. Ashok Goel pioneered the development of virtual teaching assistants,such as Jill Watson, for answering questions in online discussionforums. Jill Watson is implemented on IBM's Watson super-computingplatform and utilizes gradient descent (IbmWatson 2018), (GoelJoyner2016), (GoelJoyner 2017), (GoelPolepeddi 2017).

2.4 Application of Artificial Intelligence or Big Data Analysis toMission Planning

AI, and various versions of ML, have been applied to many fields such ascancer research, complex games like Jeopardy, Poker, and GO, and morerecently heart attack prediction with considerable success (Hutson2017). These techniques have begun to be investigated and researchedrelated to the topic of military mission planning.

The ability to learn and update asset performance models has beendemonstrated in certain mission planning domains (Ozisikyilmaz 2008),which could prove useful in learning predictive asset performancemodels, such as the ones used by the Navy. Likewise, task model learninghas been demonstrated with hierarchical task network learning (Garland2003), (Yang 2014), and explanation-based learning (Mohan 2014).

Successful mission planning requires accurate and complete models of theperformance capabilities of the assets, of the environment (includingbehavior of other agents in the environment), of the mission goals andsub-goals. Current practice for planning in situations that change is tohand-code the changes in the models of capability, environment, andgoals and then re-plan. This approach is slow and becomes infeasible inhighly dynamic situations, particularly in tactical mission planningwhere the tempo of new information requires rapid changes in the models.The data in the models may become inconsistent or obsolete faster thanour ability to hand-code new models. This can severely degrade thequality and effectiveness of automated planning aids to a degree thatthey may not be used. This problem is further exacerbated by theintroduction of unmanned assets, especially if there are frequentchanges to their sensing and autonomous capabilities. The Navy needsmethods that can rapidly and continuously plan as new information, thatprompts updating the models, becomes available (Ponirakis 2018), (Attick2018).

2.5 Application of Artificial Intelligence or Big Data Analysis toRetail Planning

In recent years, there has been significant interest in applying AI forthe purpose of customizing (personalizing) and simplifying thecustomers' shopping experience. (DennisBrown 2016) presents a system,method and user interface for providing retail salesperson(s) with asales acceleration system, using a mobile computing device, agamification system, an integration engine, that provides access toexisting corporate data, media resources communications facilities, andmanagement structures, together with a learning machine. The learningmachine selects products to offer customers, based on spend profile,multichannel sales interest, inventory and emotional state. The goalhere is to allow salesperson(s) to individualize customer productofferings with the highest predictive probability score for purchase.(Peyrelevade 2008) presents an artificial intelligence engine forproviding beauty advice. The beauty advice can include a recommendationfor a product.

In addition, there are significant commercialization activities.Invertex provides technology for analyzing customers' feet in-store orat home and suggesting what models or sizes might fit the customersbest. Oak Labs has built an interactive dressing room mirror that candisplay merchandise in different sizes and colors for shoppers, makestylist recommendations for accessorizing a look, and more. Satisfi Labsuses AI to help shoppers more easily navigate stores and other locationsto find what they are looking for. SupplyAI provides a system ofintelligence for efficient retail customer service. SupplyAI claims tobe building the world's 1st AI powered platform that helps omni-channelretailers predict and prevent returns. ReturnSense™ utilizes SupplyAI'sproprietary algorithm, for recognizing patterns in purchase behavior andpredicting the likelihood of return on every purchase made. It sends apreventative alert to retailers and validates the purchase, before theproducts are shipped (SupplyAI 2018).

2.6 Latent Semantic Indexing and Analysis

In (Yi 2009), indexing values of social tags in the context of aninformation retrieval model were assessed using a latent semanticindexing method. Socially tagged resources were classified into tenDewey Decimal Classification main classes. Then social tags assigned tothe resources were used to represent them in LSI. Similarities betweenresources were measured, and the aggregated similarities, according tothe ten DDC main classes, were compared.

3. Linkage with Automatic Assessment of Engineering Requirements

This invention can be linked with automatic assessment of engineeringrequirements, i.e., the e-design assessment from (SteingrimssonYi 2017).One can limit the automatic assessment to parameters that one canreliably extract from design tools, such as SolidWorks. To begin with,the automatic assessment may be limited to parameters such as weight,physical dimensions or surface area. Here, there is no interpretationinvolved, and hence, no chance of errors in the assessment. Othercategories, like ergonomics, may be more difficult to assessautomatically.

For the more difficult categories of engineering requirements, thisinvention provide tools and resources, along the lines of FIG. 1,enabling users to query for what they want. This tool might informdesigners

1. What is the standard size of a screw?2. What is the size of the bearing?3. What is the maximum stress?4. What is the weight?5. What is the geometry?

This type of engineering-focused querying mechanism may be very useful.We structure the automatic assessment such that it involves userinteractions. The approach for associating engineering requirements withthe categories is outlined in FIG. 2. Table 1 summarizes the 24categories listed in FIG. 2, alone with the association with the sourcefiles for detailed design:

TABLE 1 Source files corresponding to the 24 categories of engineeringrequirements. No. Category Relevant Source Files (from Design Tool) 1Aesthetics Color, Shape, Form, Texture, Finish, . . . 2 CompanyProcedures Company Requirement, Certain Standard such as GD&T or ISO9000 3 Competition Marketing 4 Cost Product, Manufacture, Tools 5Disposal Recyclable, Bio-Degradable, Green, . . . 6 Documentation LegalIssues; Litigation, Safety, Operation & Service Documents 7 EnvironmentTemperature Range, Rain, Humidity, Dust, . . . 8 Ergonomics/OperationUsers/Buyers Need, Marketing, 9 Function Users Requirement, CompanyRequirement, System/Sub-system Accomplishment 10 Installation ConnectionGeometry, Models to Install 11 Life in Service Years, Cycles, . . . 12Maintenance Professional/Government/Industry Guideline, CompanyPreference 13 Manufacturing Buyers Demand, Cost, Warranty, Marketing,Quality Assurance, . . . 14 Materials Company Guidelines, RegulationsRestrict, Marketing, Codes 15 Packaging Package Sizes, Weight, DamageResistance, Cost 16 Patents/Legal Liability Law Suits Associated withSimilar Products, Relevant Patents 17 Performance Speed, Capacity,Power, Efficiency, Accuracy, Return on Investment, . . . 18Quality/Reliability Company Warranty, Failure Rate, . . . 19 QuantityCompany Requirement, Marketing, . . . 20 Safety Government Requirement,Professional Codes and Standards, Warning Labels, Degrees of Abuse, . .. 21 Shipping Package Sizes, Weight, Damage Resistance, Distance, Cost22 Size/Volume Dimension, Volume, . . . 23 Timelines Management, CompanyRequirement, . . . 24 Weight Desired Weight, Modular, Lifting Points, .. .

As indicated above, this invention provides querying mechanism tocomplement assessment of requirements from categories that may be moredifficult to assess automatically. The querying mechanism operates atthe project or concept level. In (SteingrimssonYi 2017), the inventionutilizes queries, run at the component or assembly level, forverification of requirements that are relatively easy to assessautomatically.

In addition to the querying mechanism for helping with verification ofrequirements from categories, that are more difficult to assessautomatically, this invention is also linked with the requirementverification of (SteingrimssonYi 2017) through the definition of known,good designs. We define known, good designs as design with all thedesign requirements fulfilled (and properly verified). The automaticrequirement verification can be used to expedite the qualification ofdesigns from a (large) legacy database of known, good designs. Forfurther information, refer to the description of incremental re-trainingbelow.

The design Ecosystem of (Steingrimsson 2014), (SteingrimssonYi 2017) waspresented, in part, to provide integration across a perceived voidbetween design requirements and results.

REFERENCES

-   (Brown 2005) D C. Brown, Artificial Intelligence for Design Process    Improvement, In: Clarkson J., Eckert C. (eds) Design Process    Improvement. Springer, London, p. 158-173, 2005.-   (Alexa 2017) Wikipedia, Amazon Alexa,    https://en.wikipedia.org/wiki/Amazon_Alexa, 2017-   (Home 2017) Wikipedia, Google Home,    https://en.wikipedia.org/wiki/Google_Home, 2017.-   (GoogleAssistant 2017) Wikipedia, Google Assistant,    https://en.wikipedia.org/wiki/Google_Assistant, 2017.-   (Fuge 2014) M. Fuge, B. Peters and A. Agogino, Machine Learning    Algorithms for Recommending Design Methods, J. Mechanical Design,    136 (10) 101103, Aug. 18, 2014.-   (SteingrimssonYi 2017) B. Steingrimsson and S. Yi. Digital Ecosystem    for Engineering Team Design. U.S. patent application Ser. No.    15/613,183, filed on Jun. 3, 2017.-   (Yi 2009) K. Yi, A Study of Evaluating the Value of Social Tags as    Indexing Terms. In S. Chu, W. Ritter, & S Hawamdeh (Eds.), Managing    Knowledge for Global and Collaborative Innovations, Hackensack,    N.J.: World Scientific Publishing. Conference paper presented at the    6^(th) International Conference on Knowledge Management, Hong Kong,    December 3, p. 221-232, 2009-   (Steingrimsson 2014) B. Steingrimsson, “All-Electronic Ecosystems    for Design and Collaboration”, U.S. patent Ser. No. 14/567,516, Dec.    11, 2014.-   (Schaffernict 2011) M. Schaffernicht & S. N. Groesser, “A    comprehensive method for comparing mental models of dynamic    systems,” European Journal of Operational Research, vol. 210(1), no.    https://doi.org/10.1016/j.ejor.2010.09.003, pp. 57-67, 2011.-   (Duda 2001) R. O. Duda, P. E. Hart and D. G. Stork, Pattern    Classification, 2^(nd) edition, Wiley, 2001.-   (Salton 1975) G. Salton, A vector space model for automatic    indexing. Communications of the ACM, 18 (11), pp. 613-620, 1975.-   (Deerwester 1990) S. Deerwester, S. T. Dumais, G. W. Fumas, T. K.    Landauer and R. Harshman, Indexing by latent semantic analysis.    Journal of the American Society for Information Science, 41 (6), pp.    391, 1990.-   (Dumais 2004) S. T. Dumais, Latent Semantic Analysis, Annual Review    of Information Science and Technology, 38(1), pp 188-230, 2004.-   (Laundaucr 2007) T. Landauer, D. McNamara, S. Dennis and W. Kintsch,    Handbook of Latent Semantic Analysis, University of Colorado    Institute of Cognitive Science Series, 2007.-   (Widrow 1960) B. Widrow and M. E. Hoff, Adaptive Switching Circuits,    in IRE WESCON Convention Record, Vol. 4, New York, pp 96-104, 1960.-   (Spark 2017) Spark Developer Documentation, Spark API,    https://sparkplatform.com/docs/overview/api, 2017.-   (BigData 2017) Big Data Zone. Hadoop REST API—WebHDFS,    https://dzone.com/articles/hadoop-rest-api-webhdfs, 2017-   (Hadoop 2017) Hadoop, WebHDFS REST API,    https://hadoop.apache.org/docs/r1.0.4/webhdfs.html, 2017.-   (Kerberos 2017) Wikipedia, Kerberos (protocol),    https://en.wikipedia.org/wiki/Kerberos_(protocol), 2017.-   (Trefethen & Bau III 1997) David Bau III (1997) Numerical linear    algebra Philadelphia: Society for Industrial and Applied Mathematics    ISBN 978-0-89871-361-9, 1997.-   (DataScienceCentral 2018) Data Science Central, Artificial    Intelligence vs. Machine Learning vs. Deep Learning,    https://www.datasciencecentral.com/profiles/blogs/artificial-inteliigence-vs-machine-learning-vs-deep-learning,    2018.-   (Ozisikyilmaz 2008) Ozisikyilmaz, Memik, Choudhary, Machine Learning    Models to Predict Performance of Computer System Design    Alternatives. In Proceedings of 37th International Conference on    Parallel Processing.    cucis.ece.northwestern.edu/projects/DMS/publications/OziMem08B.pdf,    2008.-   (Garland 2003) Lcsh Garland, Learning Hierarchical Task Models by    Demonstration. Technical Report TR2003-01, Mitsubishi Electric    Research Laboratoricsk es.brandeis.edu/˜aeg/papers/garland.tr2002-04    pdf, 2003.-   (Mohan 2014) Laird Mohan, Learning Goal-Oriented Hierarchical Tasks    from Situated Interactive Instruction Proceedings of the 27th AAAI    Conference on Artificial Intelligence (AAAI),    web.eecs.umich.edu/˜soar/sitemaker/docs/pubs/mohan_AAAI_2014.pdf,    2014-   (Yang 2014) Munoz-Avila, Zhuo Yang, Learning Hierarchical Task    Network Domains from Partially Observed Plan Traces Artificial    Intelligence Journal,    cse.lehigh.edu/%7Emunoz/Publications/AIJ14.pdf, 2014-   (Ramani 2013) Karthik Ramani, Natraj Iyer, Kuiyang Lou, and    Subramaniam Jayanti, Methods, Systems, and Data Structures for    Performing Searches on Three Dimensional Objects, U.S. Pat. No.    8,429,174 B2, Granted on Apr. 23, 2013.-   (Iyer 2003) Natraj Iyer, Yagnanarayanan Kalyanaraman, Kuiyang Lou,    Subramaniam Jayanti and Karthaik Ramani, A Reconfigurable 3D    Engineering Shape Search System Part I: Shape Representation, ASME    2003 International Design Engineering Technical Conferences and    Computers and Information in Engineering Conference, 2003.-   (KuiyangLou 2003) Kuiyang Lou, Subramaniam Jayanti. Natraj Iyer,    Yagnanarayanan Kalyanaraman, Sunil Prabhakar and Karthik Ramani, A    Reconflgurable 3D Engineering Shape Search System: Part II—Database    Indexing, Retrieval, and Clustering, ASME 2003 International Design    Engineering Technical Conferences and Computers and Information in    Engineering Conference, 2003.-   (LiRamani 2007) Zhanjun Li and Karthik Ramani, Ontology-Based Design    Information Extraction and Retrieval. Artificial Intelligence for    Engineering, Design. Analysis and Manufacturing, 21. pp. 137-154,    2007.-   (SinhaBai 2016) Ayan Sinha, Jing Bai and Karthik Ramani. Deep    Learning 3D Shape Surfaces Using Geometry Images. In: Leibe B.,    Matas J., Sebe N., Welling M. (eds) Computer Vision—ECCV 2016.    ECCV 2016. Lecture Notes in Computer Science, vol 9910. Springer,    Cham, 2016.-   (BethO'Leary 2018) Mary Beth O'Leary, Revolutionizing Everyday    Products with Artificial Intelligence, MIT News,    http://digital.d2pmagazine.com/?issueID=42&pageID=51, p. 48-53,    September, 2018.-   (IbmWatson 2018) Watson, IBM website ibm.com/watson, 2018-   (GoclJoyncr 2016) A Goel and D. Joyner, An experiment in teaching    artificial intelligence online Journal for Scholarship of    Technology-Enhanced Learning 1 (1), 2016-   (GoclJoyner 2017) A. Goel and D. Joyner, Using AI to teach AI. AI    Magazine 38(2), 2017.-   (GoelPolepeddi 2017) A Goel and L Polepeddi, Jill Watson: A virtual    teaching assistant for online education Presented to the Learning    Engineering for Online Learning Workshop, Harvard University,    June 2017. To appear as a chapter in Dede, C., Richards. J. &    Saxberg, B., Editors (in preparation). Education at scale:    Engineering online teaching and learning NY: Routledge, 2017.-   (Hutson 2017) Matthew Hutson. “Self-taught artificial intelligence    beats doctors at predicting heart attacks.” Science Magazine,    sciencemag.org/news/2017/04/self-taught-artificial-intelligence-beats-doctors-predicting-heart-attacks,    Apr. 14, 2017.-   (SupplyAI 2018) AngelList, SupplyAI, https://angel.co/supplyai/,    2018.-   (Houser 2017) Nathan Houser and Ted Johnson, Data-Driven    Deployments—How Analytics Can Transform Military Positioning,    https://www2.deloitte.com/insights/us/en/industry/public-scctor/mission-analytics-military-deployments.html,    Oct. 2, 2017.-   (Ponirakis 2018) Lore-Annc Ponirakis. Learning Performance Models    and Tactical Knowledge for Continuous Mission Planning, Navy SBIR    2018.1, Topic N181-079, https://www.navysbir.com/n18_1/N181-079.htm,    2018.-   (Attick 2018) Donna Attick. Develop and Apply Artificial    Intelligence and Machine Learning Techniques for Next-Generation    Mission Planning, Navy SBIR 2018.1, Topic N181-018,    https://www.navysbir.com/n18_1/N181-018.htm, 2018.-   (JackStewart 2018) Jack Stewart, Tesla's Autopilot Was Involved in    Another Deadly Car Crash, Wired,    https://www.wired.com/story/tesla-aulopilot-self-driving-crash-california/,    Mar. 30, 2018.-   (NathanBomey 2018) Nathan Bomey, Uber self-driving car crash:    Vehicle detected Arizona pedestrian 6 seconds before accident, USA    Today,    https://www.usatoday.com/story/money/cars/2018/05/24/uber-self-driving-car-crash-ntsb-investigation/640123002/,    May 24, 2018,-   (DennisBrown 2016) Dennis E. Brown, Systems and Methods for    Artificial Intelligence-Based Gamified Retail Sales Accelerator, US    Patent Application US20180174088A1, 2016.-   (Peyrelevade 2008) Jerome Peyrelevade, Use of artificial    intelligence in providing beauty advice, U.S. Pat. No. 7,437,344 B2,    Oct. 14, 2008.-   (Hassoun 1995) M. H. Hassoun, Fundamentals of Artificial Neural    Networks, MIT Press, 1995.-   (Helwig 2017) Nathaniel E. Helwig, Multivariate Linear Regression.    University of Minnesota,    http://users.stat.umn.edu/˜helwig/notes/mvlr-Notes.pdf, 2017.-   (WikiNonlinearRegression 2018) Wikipedia, Nonlinear Regression,    https://en.wikipedia.org/wiki/Nonlinear_regression, 2018-   (WikiPolynomialRegression 2018) Wikipedia, Polynomial Regression,    https://en.wikipedia.org/wiki/Polynomial_regression, 2018.-   (WikiLogisticRegression 2018) Wikipedia, Logistic Regression,    https://en.wikipedia.org/wiki/Logistic_regression/, 2018.-   (WikiMultinomialLogisticRegression 2018) Wikipedia, Multinomial    Logistic Regression,    https://en.wikipedia.org/wiki/Multinomial_logistic_regression/,    2018.-   (WikiOrdinalRegression 2018) Wikipedia, Ordinal Regression,    https://en.wikipedia.org/wiki/Ordinal_regression/, 2018.-   (WikiFFNeuralNet 2018) Wikipedia, Feed-Forward Neural Network,    https://en.wikipedia.org/wiki/Feedforward_neural_network/, 2018.-   (OceanStudies 2000) Ocean Studies Board, Oceanography and Mine    Warfare, https://www.nap.edu/read/9773/chapter/1#ii: National    Academy Press, 2000.-   (TianWeiYaoping 2016) Z Tian, Z. Wei, L. Yaoping and P. Xianjun,    “Overview on Mission Planning System,” International Journal on    Knowledge Engineering, vol. 2, no. 1, pp. 56-59, March 2016.-   (WikiInteriorPoint 2018) Wikipedia, Interior-point method,    https://en.wikipedia.org/wiki/Interior-point_Interior-point_method/,    2018.-   (WikiNpHard 2018) Wikipedia, NP-hardness,    https://en.wikipedia.org/wiki/NP-haedness, 2018.

SUMMARY OF THE INVENTION

This invention presents a framework for utilizing big data analytics toharvest repositories of known good designs for the purpose of aidingengineering product design, mission or retail planning. We also presenta querying engine, utilizing an information retrieval framework, capableof providing teams (workforces) with information similar to the onesharvested in a database, for the purpose of improving efficiency andefficacy.

The invention outlines a novel approach for applying artificialintelligence to the training of an engineering design system model,assimilates the definition of meta-data for design containers (binders)to that of labels for books in a library, and represents customers,requirements, components and assemblies in the form of database objectswith relational dependence. We evaluate the inputs and outputs of thedesign process, but treat the process otherwise as a black box. Thedesign information can be harvested, for the purpose of improving designdecision fidelity for new designs, by providing such databaserepresentation of the design content.

Further, a querying engine (a retrieval model), that operates on thearchived design containers, and yields results that are likely tosatisfy user queries, is presented. This model, which is based on latentsemantic analysis, predicts the degree of relevance between accessibledesign information and a query, and presents the most relevant previousdesign information to the user. A few simple examples, including oneinvolving idea generation for conceptual design, are presented, in orderto provide insight into the significant utility that may be derived fromthe retrieval model.

Along these lines, this invention presents the framework in FIG. 1, forutilizing big data analytics to harvest repositories of known gooddesigns for the purpose of assisting with specific areas in engineeringproduct design. While the framework can be applied to product design ingeneral, as well as to mission and retail planning, mechanical productdesign is here first considered (in FIG. 1). The framework in FIG. 1 forpredictive analytics assumes that, during the course of design projects,design information is captured in structured fashion using software suchas the Ecosystem (SteingrimssonYi 2017), (Steingrimsson 2014). Projectbinders from past design projects are then archived in databases andmade available to designers working on new design projects. These designcontainers were referred to as e-design notebooks (SteingrimssonYi2017), (Steingrimsson 2014). The system is trained so that it canprovide the best possible guiding information, such as for new productdesign, and sanitize the decisions made on the new projects. On the newprojects, the system helps identify anomalies (causes of concern),defined in terms of deviations from the guiding reference, and promptfor investigation.

The benefits associated with the big data framework are multifold:

-   1. By comparing new design content against the guiding designs    (reference), the fidelity of decisions related to the new design can    be improved, as indicated above.-   2. Through deployment of latent semantic analysis, the big data    framework can process a variety of user queries, retrieve the most    relevant archived information, and present to the user.    -   A simple example involving idea generation (brainstorming) for        Concept Design is presented in order to provide insight into the        significant utility that may be derived from the invention.-   3. While FIG. 16 and FIG. 18 present a relatively simple example, to    convey the concept, more nuanced examples can be crafted around the    Detailed Design phase.

Depending on users' needs, the big data framework can query forinformation related to specific standards, regulations, policies,customer information, internal requirements, best practices, previoussolutions, analogies, material properties, common components, etc.,retrieve information from the databases yielding the best match, andpresent to the user.

By harvesting information from past design projects, including designsthat eventually have not received selection for implementation, but maystill fulfill the design requirements, for the purpose of providingpredictions aimed at aiding future design projects, we think we may betaking a step towards a paradigm shift in engineering design. Thisinvention applies predictive analytics to large archives from pastdesign projects, for the purpose of helping designers on future designprojects.

This invention presents a similar big data analytics framework formilitary mission planning. The framework assumes training of anautomated mission planning system using requirements, combined withtactical mission plans or asset performance models from past missionplanning projects. When applying requirements from a new missionplanning project as input, the trained system offers a guiding plan asan aid to mission designers. We show how this guiding plan can leverageand exploit mission performance data and user feedback, including afteraction reports, planning decisions, and critiques of system performance.Given archived data from past missions involving weapon selection,determination of waypoint, fuel usage calculation, time line developmentand communication planning, the invention assumes training of machinelearning tools, followed by application of corresponding inputs for newmissions, for the purpose of generating references decisions aids forthe new missions.

Extension to retail planning is also presented.

DESCRIPTION OF THE DRAWINGS

FIG. 1 outlines the framework for applying predictive analytics tospecific areas of engineering product design, such as mechanical design.The same framework can be extended to other areas of engineering design.The input requirement vector (“features desired”) is expected to staylargely the same, but the composition of the output vector (“featuresobserved”) will likely be revised in accordance with the new output.

FIG. 2 presents a generic method for associating data files fromDetailed Design with the engineering requirements. Such a method isneeded, for the purpose of informing the automatic assessment which datafiles to look at in order to obtain information needed for assessment ofgiven engineering requirements.

FIG. 3 presents the schematics of a logistic regression classifier inthe form of a single-layer feed-forward neural network.

FIG. 4 offers a neural network representation of Kolmogorov's theorem. Auniversal transformation M maps R_(n) into several uni-dimensionaltransformations (Hassoun 1995). We draw upon Kolmogorov's theorem toillustrate generality of our system model. Kolmogorov's theorem providestheoretical foundation for the capabilities of two-layer feed-forwardneural network in terms of approximating any continuous function.

FIG. 5 presents schematics of a multi-layer perceptron network.

FIG. 6a and FIG. 6b provides visualization of the convex nature of themultivariate linear regression problem, for deriving the system model,when one or more input requirements are limited to a single continuousrange (a range limited by a minimum and a maximum value).

FIG. 7 highlights parallels between the strike mission planning processand the design process. The user interface for multi-strike missionplanning system may be crafted through minor modification of theEcosystem interface for engineering design.

FIG. 8 highlights the cost savings that can be derived from earlyidentification of design oversights, in the context of the V-model forsystem engineering.

FIG. 9 illustrates how the Ecosystem design framework can providedecision support across mission domains.

FIG. 10 presents key elements from a generic retail management process.

FIG. 11 presents further specifics for a generic inventory managementsystem. The big data framework can help with the forecasting of categorysales (Step 1).

FIG. 12 provides an example of customer database objects. This drawingis adapted from (SteingrimssonYi 2017), but with additional contextprovided. The database objects for mechanical product design, along withtheir corresponding attributes, are defined based on function.

FIG. 13 offers an example of a requirement database object, togetherwith its relational association with a customer object. Here we assume,for simplicity, that a given requirement originates from a singlecustomer.

FIG. 14a and FIG. 14b provides an illustration of a design database withassembly and component objects, together with its relational associationwith a requirement object. Here we assume that a component or assemblyobject can address many requirements, and that given requirement mayappear in many components or assemblies. Again, the database objects formechanical product design, along with their corresponding attributes,are defined based on function.

FIG. 15 illustrates how a master assembly object (a Level 0 object),along with the requirements that it addresses, can be represented inrelation with sub-assemblies and components (Level 1 objects), alongwith the requirements that the sub-assemblies or components address.FIG. 15 similarly shows how the Level 1 objects can be represented inrelation to corresponding Level 2 objects (together with therequirements that the Level 2 objects address). FIG. 15 further providesan illustration of how component options and risks, for an overalldesign, can be programmed into databases based on existing engineeringknowledge (for example, machine design text awareness of risks forcertain components or uses). Analogous to FIG. 12-FIG. 14, this drawingis replicated from (SteingrimssonYi 2017), but with additional contextprovided.

FIG. 16 provides an illustration of the process of retrieving previouse-design history relevant to a user query. We refer to this process asthe querying engine. FIG. 16 also clarifies the context between thequerying engine and the predictive analytics framework in FIG. 1.

FIG. 17 illustrates how the big data framework can be applied to theConcept Design stage of a project involving the design of asingle-person Go Kart lift stand.

FIG. 18 presents detailed explanations related to application of theprocess of retrieving previous e-design history relevant to the userinput query from FIG. 16.

FIG. 19 outlines the high-level framework for applying predictiveanalytics to automated planning of aerial strike missions.

FIG. 20 presents detailed explanations related to application of theprocess of retrieving previous e-design history relevant to an inputquery from an aerial strike mission planner (an input query associatedwith FIG. 19).

FIG. 21 describes application of the framework for predictive analyticsto automated surface or underwater mission planning.

FIG. 22 presents detailed explanations related to application of theprocess of retrieving previous e-design history relevant to an inputquery from a surface or underwater mission planner (an input queryassociated with FIG. 21).

FIG. 23 presents application of the framework for predictive analyticsto retail planning and supply chain management.

FIG. 24 presents the machine learning framework as a high-levelframework of learning methods overlooking JMPS. The big data frameworkprovides advice to JMPS, based on outcomes and performance dataprovided.

For the case of surface or underwater mission planning, FIG. 25 presentsthe big data framework as a high-level framework, containing machinelearning methods that interface with the Shipboard MEDAL, the CMWCMEDAL, and the data warehouse.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 1. Definitions

Table 2 captures the primary acronyms used in the patent.

TABLE 2 Summary of the primary definitions and acronyms. Name DefinitionAI Artificial Intelligence CAG Commander, Air Group CNN ConvolutionalNeural Network DCC Dewey Decimal Classification DL Deep Learning DODDepartment of Defense FLOP Floating Point Operation IR InformationRetrieval JMPS Joint Mission Planning System LIDAR Light ImagingDetection And Ranging LMS Lease Mean Square LSA Latent Semantic AnalysisLSI Latent Semantic Indexing MEDAL Mine Warfare Environmental DecisionAid Library ML Machine Learning OEM Original Equipment Manufacturer PDFProbability Density Function PDS Product Design Specification PLMProduct Lifecycle Management SLAM Simultaneous Localization and MappingSVD Singular Value Decomposition SW Software

We define artificial intelligence as the use of computers to mimic thecognitive functions of humans. When machines carry out tasks based onalgorithms in an “intelligent” manner, that is AI. Artificialintelligence is a broader concept than machine learning(DataScienceCentral 2018).

We define machine learning as a subset of AI that focuses on the abilityof machines to receive a set of data and learn for themselves, andchange algorithms as they learn more about the information that they areprocessing (DataScienceCentral 2018).

We refer to deep learning as a subset of machine learning. We definedeep learning in terms of “deep neural networks”, i.e., neural networkscomprising of two or more layers. Deep learning networks need to seelarge quantities of items in order to be trained (DataScienceCentral2018).

We define a “known, good design” as a design that has satisfied all therequirements. This is a design that has the requirement vector fullyspecified, and where the design produced (the output vector) fulfillsthe design requirements.

Supervised learning is a data mining task that involves inference of afunction from labeled training data.

Unsupervised learning is a type of machine learning algorithm used todraw inferences from datasets consisting of input data without labeledresponses.

Reinforcement learning is an area of machine learning concerned with howsoftware agents ought to take actions in an environment so as tomaximize some notion of cumulative reward.

2. Best Mode of the Invention

FIG. 1, FIG. 14, FIG. 15, FIG. 16, FIG. 19, FIG. 21 and FIG. 23 capturethe best mode contemplated by the inventors, according to the conceptsof the present invention.

3. The Generic System Model Assumed

We assume a generic system model:

{tilde over (y)}=ƒ({tilde over (x)}).  (1)

The input vector, {tilde over (x)}, could be considered as consisting ofdesign variables (design targets or requirements). The transformation,ƒ( ) could be a non-linear function of the input, {tilde over (x)}.Engineers transfer the requirements, {tilde over (x)}, into the product,{tilde over (y)}, through the transformation. The transformation, ƒ( ),may contain reasoning and knowledge on how to make {tilde over (y)} (onhow to design or produce {tilde over (y)}). To other extent, we treatthe making of {tilde over (y)} from {tilde over (x)} as a black box. Inthe case of the design process, the transformation, ƒ( ), may consist ofcustomers, requirements, systems and assembly. We present artificialintelligence and supervised learning as one of the options to train thesystem, as explained below.

The input vector, {tilde over (x)}, could capture design criteria, suchas the desired weight, width, height and length of an automotive part(the “intended features”). The elements of the product vector, {tildeover (y)}, could capture performance of the finalized part, or evenideas or options relevant to specific design stages (the “observedfeatures”). The elements of {tilde over (x)}, and even {tilde over (y)},could be derived from the 24 categories listed in Table 1.

For clarification, refer to the examples below. It is assumed that theorganization adopting the invention has practiced structured capture ofthe binders ({tilde over (x)}, {tilde over (y)}) from past projects, forexample in SW like the Ecosystem (SteingrimssonYi 2017), (Steingrimsson2014). The binders from the past projects may, for example, have beenarchived in an internal database.

This invention offers a scalable solution, depending on the size of theinput data. In the case of small (design) databases, we presentregression as a suitable tool for determining the system model. Forlarge (design) databases, say, hundreds, thousands or millions of({tilde over (x)}, {tilde over (y)}) duplets, we present neural networksas a suitable tool for determining the system model.

3.1 Multivariate Linear Regression

In case of archived binders of relatively small-to-modest size, or evenof moderately large size, we recommend applying a multivariate(multiple) linear regression model: Drawing upon FIG. 1, we model thearchived binders, in this case, as

{tilde over (y)} _(ik) =b _(0k)+Σ_(j=1) ^(p) b _(jk) {tilde over (x)}_(ij) +e _(ij)  (2)

for i∈{1, . . . , n} and k∈{1, . . . , m}. Here,

-   -   {tilde over (y)}_(ik)∈        is the k-th real-valued response for the i-th observation.    -   b_(0k)∈        is the regression intercept for k-th response.    -   b_(jk)∈        is the j-th predictor's regression slope for k-th response.    -   (e_(i1), . . . , e_(im))˜N(0_(m),Σ) is a multivariate Gaussian        error vector.

The archived binders can be stacked up into a matrix and represented as

$\begin{matrix}{\begin{bmatrix}{\overset{\sim}{y}}_{11} & \cdots & {\overset{\sim}{y}}_{1m} \\{\overset{\sim}{y}}_{21} & \cdots & {\overset{\sim}{y}}_{2m} \\{\overset{\sim}{y}}_{31} & \cdots & {\overset{\sim}{y}}_{3m} \\\vdots & \ddots & \vdots \\{\overset{\sim}{y}}_{n\; 1} & \cdots & {\overset{\sim}{y}}_{nm}\end{bmatrix} = {{\begin{bmatrix}1 & {\overset{\sim}{x}}_{11} & {\overset{\sim}{x}}_{12} & \cdots & {\overset{\sim}{x}}_{1p} \\1 & {\overset{\sim}{x}}_{21} & {\overset{\sim}{x}}_{22} & \cdots & {\overset{\sim}{x}}_{2p} \\1 & {\overset{\sim}{x}}_{31} & {\overset{\sim}{x}}_{32} & \cdots & {\overset{\sim}{x}}_{3p} \\1 & \vdots & \vdots & \ddots & \vdots \\1 & {\overset{\sim}{x}}_{n\; 1} & {\overset{\sim}{x}}_{n\; 2} & \cdots & {\overset{\sim}{x}}_{np}\end{bmatrix}\begin{bmatrix}b_{01} & \cdots & b_{0m} \\b_{11} & \cdots & b_{1m} \\b_{21} & \cdots & b_{2m} \\\vdots & \ddots & \vdots \\b_{p\; 1} & \cdots & b_{pm}\end{bmatrix}} + {\quad\begin{bmatrix}e_{11} & \cdots & e_{1m} \\e_{21} & \cdots & e_{2m} \\e_{31} & \cdots & e_{3m} \\\vdots & \ddots & \vdots \\e_{n\; 1} & \ldots & e_{nm}\end{bmatrix}}}} & (3) \\{\mspace{76mu} {\overset{\sim}{Y} = {{\overset{\sim}{X}B} + {E.}}}} & (4)\end{matrix}$

The ordinary least squares problem is

min_(BER(p+)xm) ∥{tilde over (Y)}−{tilde over (X)}B∥ ²  (5)

where ∥⋅∥ denotes the Frobenius norm (Helwig 2017).

The ordinary least squares problem has a solution of the form (Helwig2017)

{tilde over (B)}=({tilde over (X)} ^(T) {tilde over (X)})⁻¹ {tilde over(X)} ^(T) {tilde over (Y)}.  (6)

A new project will give rise to the input vector x from which theguiding reference y is obtained as

y={circumflex over (B)} ^(T) x.  (7)

Note that the multivariate regression offers a deterministic solution(no convergence problems).

3.2 Other Regression Analyses Available

While multivariate linear regression may be a natural choice, in caseregression analysis is preferred for determining the system model, inparticular for continuous and real-valued inputs, there are otheroptions available. These include:

-   1. Polynomial regression (WikiPolynomialRegression 2018).-   2. Logistic regression (WikiLogisticRegression 2018).-   3. Multinomial logistic regression    (WikiMultinomialLogisticRegression 2018).-   4. Ordinal regression (WikiOrdinalRegression 2018).-   5. Other types of nonlinear regression (PennState 2018),    (WikiNonlinearRegression 2018).

3.3 AI Predictor (Neural Network)

Neural networks are typically preferred over linear regression in caseswhen the input data size is very large. Neural networks may be be usedfor image classification, for example to detect an event or artifact ina sample of images, each of which may contain approx. 10⁶ pixels, eachwith, say, 64 color values. Neural networks are similarly suitable foridentification of events in sampled audio, where the input may occupygigantic space of time and frequency. Neural networks may be preferred,in the case of a very large set of archived binders.

Neural networks currently used in machine learning include Feed-ForwardNeural Networks, Radial Basis Function Neural Networks, KohonenSelf-Organizing Neural Networks, Recurrent Neural Networks,Convolutional Neural Networks and Modular Neural Networks. Other neuralnetworks supported by this invention include Deep Feed Forward Networks,Long/Short Term Memory, Gated Recurrent Units, Auto Encoders,Variational Auto Encoders, Denoising Auto Encoders, Sparse AutoEncoders, Markov Chains, Hopfield Networks, Boltzmann Machines,Restricted Boltzmann Machines, Deep Belief Networks, Deep ConvolutionalNetworks, Deconvolutional Networks, Deep Convolutional Inverse GraphicsNetworks, Generative Adversial Networks, Liquid State Machines, ExtremeLearning Machines, Echo State Networks, Deep Residual Networks, KohonenNetworks, Support Vector Machines and Neural Turing Machines.

Even so, the main emphasis here is on Single and Two-Layer Feed-ForwardNeural Networks. We show how Two-Layer Feed-Forward Neural Networks canapproximate any system model.

For simplicity, it is assumed that input parameters take on values froma continuous range.

1. Single-Layer Feed-Forward Neural Network

In a feed-forward neural network, the connections between the nodes donot form a cycle. The simplest kind of neural network is a single-layerperceptron network, which consists of a single layer of output nodes;the inputs are fed directly to the outputs via a series of weights. Thesum of the products of the weights and the inputs is calculated in eachnode, and if the value is above a given threshold (typically 0) theneuron fires and takes on the activated value (typically 1). Otherwise,it takes on the deactivated value (typically −1). Perceptrons can betrained by a simple learning algorithm that is usually referred to asthe Delta Rule (see (16)). It calculates the errors between calculatedoutput and sample output data, and uses this to create an adjustment tothe weights, thus implementing a form of gradient descent. Single-layerperceptrons are only capable of learning linearly separable patterns.

A common choice for the activation function is the sigmoid (logistic)function:

$\begin{matrix}{{f(z)} = {\frac{1}{1 + e^{- z}}.}} & (8)\end{matrix}$

With this choice, the single-layer network shown in FIG. 3 is identicalto logistic regression.

A single-layer neural network has guaranteed convergence (equivalent toregression).

2. Multi-Layer Feed-Forward Neural Network

It may be recognized that a straight forward application of asingle-layer neural network model cannot accommodate all requirements.To handle binary requirements (simple presence or absence), XOR-likeconditions, or categorical requirements, a two-layer neural network maybe necessary (Duda 2001).

A multi-layer feed-forward neural network, shown in FIG. 5, consists ofmultiple layers of computational units, interconnected in a feed-forwardfashion. Each neuron in one layer has directed connections to theneurons of the subsequent layer. In many applications the units of thesenetworks apply the sigmoid function (8) as the activation function(WikiFFNeuralNet 2018). Similar to the single-layer case, the activationis defined as

a _(j) ^(i)=ƒ(Σ_(k) w _(ik) x _(kj) +b _(i)).  (9)

Convergence of a multi-layer neural network involves non-convexoptimization, and hence is not guaranteed. You can get stuck in localminima.

3.4 Approximation Capabilities of Feed-Forward Neural Networks forContinuous Functions

According to Kolmogorov, any continuous, real-valued function can bemodeled in the form of a two-layer neural network. More specifically,Kolmogorov showed in 1957 that any continuous real-valued function ƒ(x₁,x₂, . . . , x_(n)) defined on [0,1] n, with n≥2, can be represented inthe form

y=ƒ(x ₁ ,x ₂ , . . . , x _(n))=Σ_(j=1) ^(2n+1) g _(j)(Σ_(i=1)^(n)ϕ_(ij)(x _(i)))  (10)

where the g_(j)'s are properly chosen functions of one variable, and theϕ_(ij)'s are continuously monotonically increasing functions independentof ƒ. FIG. 4 offers a neural network representation of Kolmogorov'stheorem (Hassoun 1995).

While users are welcome to use neural networks of three or more layers,we recommend limiting the networks to two layers, for complexity sake.Yet, even with two layers, this invention can approximate a genericsystem model, per Kolmogorov's theorem.

3.5 Multivariate Linear Regression when the Input Requirements Definedin Terms of Ranges

When one or more of the inputs is limited to a single, continuous range

x _(j)∈[x _(j,min) ,x _(j,max)]  (11)

the multivariate linear regression problem can be formulated as

$\begin{matrix}{\min_{\begin{matrix}{B \in R^{{({p + 1})}{xm}}} \\{\overset{\sim}{X} \in {{Conv}{(P)}} \subseteq R^{({p + 1})}}\end{matrix}}{{{\overset{\sim}{Y} - {\overset{\sim}{X}B}}}^{2}.}} & (12)\end{matrix}$

Here, Conv(P)⊆R^((p+1)) is the convex hull defined by the (p+1) elementscomprising the vector {tilde over (X)}.

It is important to recognize that when any given element of the inputvector {tilde over (X)} is confined to a single, continuous range, theresulting vector subset forms a convex polytope in R^((p+1)). FIG. 6illustrates, for a simple case, that you can travel between any givenpoints in the polytope, and yet stay within the set.

While the optimization problem in (12) may not have a closed-formsolution, the fact that it is convex means that you can generate asolution using efficient interior-point solvers, with polynomialworst-case complexity (WikiInteriorPoint 2018):

O((p+1)^(3.5)).  (13)

3.6 Assigning Importance Levels (Priorities) to Specific Requirements

The optimization problem in (12) can be extended not only to supportranges, but also priorities assigned to specific requirements:

$\begin{matrix}{\min_{\begin{matrix}{B \in R^{{({p + 1})}{xm}}} \\{\overset{\sim}{X} \in {{Conv}{(P)}} \subseteq R^{({p + 1})}}\end{matrix}}{{{\overset{\sim}{Y} - {\overset{\sim}{X}\overset{\sim}{W}B}}}^{2}.}} & (14)\end{matrix}$

Here {tilde over (W)} is a p×p diagonal matrix with element w_(jj)specifying the priority associated with input requirement j, x_(j). Therequirements with higher priority receive higher weight. Theoptimization problem is still linear and convex.

3.7 Sensitivity Analysis

In case it is of interest, the interior-point methods, that can be usedto solve the optimization problems in (12) and (14), can also inform thedesigners of the relative contributions of given input requirements tothe objective function. The sensitivity to a given input requirement isdefined as the derivative of the objective function with respect to thatrequirement. The sensitivities come about as Lagrange multipliers thatare produced as a by-product from the interior-point solvers. While youalready may be hitting a boundary, the derivatives may inform designersabout variables that still can change within the feasible set.

3.8 Input Requirements Defined in Terms of Categories

Once an input requirement consists of categories, the optimizationproblem automatically becomes NP-hard, meaning the run time of a solveris no longer guaranteed to be deterministically polynomial in the sizeof the problem. For categorical (discrete) input, there may be casesthat can be solved quickly. But in general the problem may be subjectedto exponential worst-case complexity (WikiNpHard 2018).

4. The Generic Processes Assumed 4.1 Design Process

It is assumed that a classical design process consists of the followingstages:

-   -   Requirement Gathering→Concept Design→Detailed Design→Final        Design.

Such process is modeled in the Ecosystem SW (SteingrimssonYi 2017),(Steingrimsson 2014). The customers, customer requirements andcorresponding engineering requirements are defined, as a part of theRequirement Gathering, and captured in the Product Design Specification.The Concept Design consists of brainstorming, concept design analysis(scoring) and design selection. The Detailed Design may capture detailedanalysis of both the overall system and associated subsystems. FinalDesign is usually preparation for prototype building or production, andmay include steps such as testing and requirement validation(SteingrimssonYi 2017), (Steingrimsson 2014).

4.2 Strike Mission Planning Process

1. Similarities with the Design Process

Mission and strike planning are complex processes, integrating specificperformance characteristics for each platform into a comprehensivemission.

As FIG. 7, Table 3 and Table 4 illustrate, the Navy's tactical aircraftstrike planning process provides high degree of resemblance with theengineering design process presented in (Steingrimsson 2014) and(SteingrimssonYi 2017). Hence, the Ecosystem interface of (Steingrimsson2014) and (SteingrimssonYi 2017) can be applied to the strike planningprocess, with relatively modest alterations.

TABLE 3 Similarities between the design process and the mission strikeplanning process. Design Process Strike Planning Cycle RequirementGathering 1. Receive tasking 2. Task strike teams Concept Design 3.Brainstorm rough plan 4. Brief CAG Detailed Design 5. Create detailedplan 6. Conduct briefings Final Design 7. Execute the mission 8. Gatherbomb damage assessment

TABLE 4 Similarities between the Ecosystem software for engineeringdesign and the Navy strike mission planning. Ecosystem Navy StrikeMission Planning Design process Tactical air craft strike planningprocess Design decision Planning decision Design project Combat search &rescue mission Requirement gathering Mission planning Project sponsorNational Command Authority, Joint Chiefs of Staff, Commanders in Chief(CAG) Supervisor or instructor Strike leader Design team formationStrike team formation Ecosystem: High-level SW JMPS: High-level SW forjoint planning for design process

2. Differences

-   1. The typical Military Personnel Center conducts mission planning    from start to finish in an approx. 8 hour window, whereas    engineering design projects are usually completed over a period of    consisting of a few to several months or even years.-   2. In combat, the crew that plans is typically not the crew that    flies it, so decisions are sometimes made to enable as much    flexibility as possible.-   3. In combat, the crew that plans is typically NOT the crew that    executes it, so decisions are sometimes made to enable as much    flexibility as possible and delegate the decision authority to the    lowest possible level (i.e. the person in the cockpit in the    mission).-   4. In significant contrast to the goal of a design process, the end    result of a mission is usually fairly different from the way it was    envisioned in the plan. This is due to the fact that the enemy has a    vote in how the fight takes place, and you are dealing with severely    imperfect assumptions that affect the battle space. The input    assumptions may be subjected to considerable time dependence.

4.3 Surface and Underwater Mission Planning

1. Similarities with the Design Process

The Ecosystem supports the “V” model of system engineering, shown inFIG. 8, and aligns with Navy's thrust in the area of Surface WarfareMission Engineering and Analysis. In addition to providing decisionsupport for all phases in the design process, FIG. 9 and Table 5 showhow the Ecosystem can also support each of the Navy mission domains.

TABLE 5 Similarities between the design process and the Navy missiondomains. Design Process Navy Mission Domain Requirement Gathering PlanConcept Design Detect Detailed Design Control Final Design Engage Assess

2. Differences

-   1. Just as in the case of the strike mission planning process, a    Military Personnel Center may conduct mission planning from start to    finish in an approx. 8 hour window, whereas engineering design    projects may be completed over a period of consisting of a few to    several months or even years.-   2. Furthermore, in combat, the personnel that plans is typically not    the personnel that executes the mission, so decisions are often made    with flexibility in mind.-   3. In severe contrast to the goal of a design process, the end    result of a mission may differ significantly from the way it was    envisioned in the plan. This is due to the fact that the enemy has a    vote in how a fight takes place, and you may be dealing with    imperfect assumptions that affect the battle space.-   4. Tactics may need to be adapted to account for changes in    environmental conditions.

4.4 Retail Planning Process

FIG. 10 and FIG. 11 summarize a retail planning process. Table 6presents a mapping between the retail planning processing and the designprocess.

TABLE 6 Mapping between the design process and the retail planningprocess. Design Process Retail Planning Process Requirement Gathering 1.Forecast category sales 2. Develop an assortment plan Concept Design 3.Determine appropriate inventory levels and product availability 4.Develop a plan for managing the inventory Detailed Design 5. Allocatemerchandise for the stores 6. Buy merchandise Final Design 7. Monitorand evaluate performance and make adjustments

We apply the big data framework to the retail planning process byharvesting analogies from Table 6, in a similar fashion as we harvestedanalogies from Table 3 and Table 5, in order to extend the big dataframework from the design process to the mission planning process. Tothis effect, we define the input and output vectors as follow:

x=[Forecasted_sale_category1_store1,Forecasted_sale_category2_store1, .. . , Forecasted_sale_category1_store2,Forecasted_sale_category2_store2,. . . ,Forecasted_sale_category1_store3,Forecasted_sale_category2_store3, . . ., . . . ]   (15)

y=[Observed_sale_category1_store1,Observed_sale_category2_store1, . . ., Observed_sale_category1_store2,Observed_sale_category2_store2, . . . ,Observed_sale_category1_store3,Observed_sale_category2_store3, . . . , .. . ]   (16)

6. How to Make the Invention 6.1 Archived Binders

The archived project binders in FIG. 1, FIG. 19, FIG. 21 and FIG. 23consist of past project binders, and are taken to represent known gooddesigns, known good mission plans, or known good retail plans (projectswith all requirements fulfilled). The input vector, {tilde over (x)}, inthe ({tilde over (x)}, {tilde over (y)}) duplet could capture designcriteria, such as the desired weight, width, height and length of anautomotive part (the “intended features”). The elements of the productvector, {tilde over (y)}, could capture performance of the finalizedpart, or even ideas or options relevant to specific design stages (the“observed features”), as noted above. The duplets are used to train ageneric system model, per (3) and (4).

While the composition of the ({tilde over (x)}, {tilde over (y)})duplets varies between applications, {tilde over (x)}, typicallyrepresents some type of ‘input’, in the case of an engineeringapplication the complete ‘requirement list’, but {tilde over (y)} the‘output’ of a ‘design’ or a ‘plan’. In the case of mechanical design,{tilde over (y)} could represent a complete parameterized list of theoverall assembly from the solid modeling tool of choice. {tilde over(y)} could also contain information related to bill of materials ordrawings.

The project binders may contain pointers to pertinent content, based ondesigner inputs and available information. The input format captures andpreserves content and associates with the relevant context. Thisfacilitates storage for future use. Pertinent third-party data isaccessed from databases with available context provided. The databasesmay be owned by the vendor of the SW design tool used, by a customer orby a third party. Designers ultimately choose to consider theinformation that is most relevant for any given design decision. Thisarrangement allows designers to leverage digital content management tomake more informed design decisions without losing focus of the primarydesign challenge.

The information developed for the project binders in FIG. 1 consists ofpointers to the PDS and design (database) objects. The PDS comprises ofrequirement objects, in programming context, and the design objects arecomprised of component and assembly objects. Both can have hierarchyimposed. The design data itself is stored in mass outside theapplication.

6.2 New Predictions

For new designs, designers could extract the design vector, x, in FIG. 1from the new requirements, apply to the system model, and get theguiding design, y, as an output. The guiding design, y, could be areference (starting point) for design of the new product. Such referencemay help improve the fidelity of design decisions. If design decisionscause the product to deviate significantly from the reference, y,explanations are likely warranted.

Similarly, this invention assumes training of an automated missionplanning system using requirements, combined with tactical mission plansor asset performance models from past mission planning projects, asshown in FIG. 20 and FIG. 22. When applying requirements from a newmission planning project as input, the trained system offers a guidingplan as an aid to mission designers. This guiding plan can leverage andexploit mission performance data and user feedback, including afteraction reports, planning decisions, and critiques of system performance.

When designing something specific, like a bolt, one expects narrowlydefined requirements and relatively good prediction capabilities. As thesystem model captures a broader subject, one expects more variations inthe model, and worse prediction capability.

6.3 Identification of Anomalies

Identification of anomalies (significant deviations from the referenceprediction) depends on the nature of the data (categorical orcontinuous). In general, the following methods apply:

-   1. One can compare distribution of each variable either by using    quantiles or another statistical test, to see if the variations are    significantly different.-   2. One can count occurrence of each label or category and compare.-   3. One can employ a distance measure, such as the Mahalanobis    distance, and look for big changes.-   4. One can apply an absolute difference between new and old data,    set a threshold, and report everything that exceeds the threshold.-   5. One can apply a multidimensional technique, like correlation    matrix, principal components, clustering, etc., and look for    changes.-   6. One can employ statistical/ML models specialized for anomaly    detection, including Support Vector Machines, t-distributed    Stochastic Neighbor Embedding, Isolation Forests, Peer Group    Analysis or Break Point Analysis.

6.4 Database Objects

The database objects suitable for engineering product design, mission orretail planning are defined along with their associated attributes. Bydefining the databases based on function, four databases with seeminglyreasonable attributes are proposed. The database management overheadassociated with the proposed architecture is expected to be minimal.

1. Customer and Requirement Objects

FIG. 12 and FIG. 13 present embodiments of customer and requirementobjects from a database containing the PDS objects. Through the PDS, thedesigner builds up a collection of pointers to pertinent designinformation objects. It is of key importance to define proper attributesfor the object pointers in the PDS database, and formulate metadata andleading indices (index maps) accordingly. For the PDS database, theobject pointers considered pertinent are listed in Table 7 and Table 8.The constraints in Table 8 may be binary and can be relatively easy toverify. The performance requirements typically involve binarythresholds, and are judged in accordance to design performance relativeto the threshold. The objectives involve no thresholds, but ratherprovide optimization considerations for decisions.

TABLE 7 Essential attributes pertinent to the customer objects in thePDS database. Attribute Description CustomerID Unique ID for thecustomer object Name Organization, Person, Entity Type Internal,External, Other Importance Low, Medium, High

Note the requirements are not listed in the customer objects, but thecustomers are listed in the requirement objects. This avoidance ofduplicity, and cyclic relationships, is consistent with the designphilosophy behind relational databases.

The framework for predictive analytics is capable of generating,managing, and presenting content with relevance to the design problem athand in the databases available. It is assumed that, during the courseof a design project, the database continues to grow. If design contentis not readily available through a third-party or in-house, designersare apt to define it.

TABLE 8 Essential attributes pertinent to requirement objects in the PDSdatabase. Attribute Description RequirementID Unique ID for therequirement object Name Descriptive name for the Requirement objectOwner Key to customer database: Customer[i] Importance Low, Medium, HighType Constraint, Performance or Objective Function Function (e.g.,mechanics) that requirement is addressing Characteristics Key tocharacteristics database: Characteristic[j] Could be based on thefunction Units Key to units database: Units[k] Threshold Value forbinary assessment

FIG. 1 shows how the PDS object can be built using pointers to adatabase, for the purpose of being big data compatible.

2. Assembly and Component Objects

FIG. 14 presents an embodiment of the assembly and component objectsfrom the design database. The assembly objects consist of nested,aggregated subordinate levels, and have authority to define requirementsapplicable to the subordinates. The component objects consist ofindividual parts, pieces, or obtainable, self-contained assemblies. Incase of the Design database, the pertinent attributes for the assemblyand component objects are listed in Table 10 and Table 9. The rules inTable 10 specify the governing constraints of aggregated subassembliesand components. It is assumed that the design database complies withstandard relational database (schema) formats for big datacompatibility.

In FIG. 14, we assume that a component or assembly object can addressmany requirements, and that given requirement may appear in manycomponents or assemblies. The relational dependence between the assemblyor component objects and the requirement objects is configuredaccordingly. Again, the database objects for mechanical product design,along with their corresponding attributes, are defined based onfunction.

TABLE 9 Attributes pertinent to components objects in the designdatabase (SteingrimssonYi 2017). Attribute Description ComponentIDUnique ID of the component object Name Descriptive name for the Assemblyobject Requirements Key to the PDS database: Requirement[ ] Input Key todatabase: Flow[ ] Output Key to database: Flow[ ] Process Key todatabase: Process[ ] Dimensions Nominal and tolerance, in the form ofsolid model data Material Key to database (for material properties):Material[ ] Properties Description of miscellaneous properties

TABLE 10 Attributes pertinent to assembly objects in the design database(SteingrimssonYi 2017). Attribute Description AssemblyID Unique ID ofthe assembly objects Name Descriptive name for the Assembly objectRequirements Key to the PDS database: Requirement[ ] Subordinates Definesubassemblies and components Input Key to database: Flow[ ] Output Keyto database: Flow[ ] Process Key to database: Process[ ] Rules Key toRules database: Rules[ ]

3. Mission Planning Objects

We apply the framework for predictive analytics to generate missionplans in near-autonomous mode, given the current work flow for missionplanning, by leveraging data, models and standards in the core databasefor the mission planning system. The framework is capable of generating,managing, and presenting content with relevance to the mission plan athand in the databases available. We define database objects suitable formission planning, along with their associated attributes, based on thefunction desired.

Specifically, for multi-domain, multi-asset mission planning, the key toapplying the framework for predictive analytics involves “correctly”collecting data into the database and tagging. All the domains andassets supported should have a universal index, and the indices shouldbe recorded as a part of the Requirement objects, as shown in Table 11.There may be a “local” versions of the database, for each domain and/orasset. As you collect data into the “local” versions of the database,you may tag the data to indicate which domain or asset it correspondsto. Then you may have a “global”, centralized database for all theassets and domains.

TABLE 11 Selected items from requirement objects for mission planning.Item Explanation Goals The mission goals Location Location of the targetTarget Target selected High-payoff targets The high-payoff targetsDomain To enable multi-vehicle mission planning (universal indexing ofthe domains) Asset To enable multi-domain mission planning (universalindexing of the assets) Security level To account for different securitylevels (standardized names of levels)

4. External Databases

As shown in FIG. 1, FIG. 19, FIG. 21 and FIG. 23, the design contentaggregated is multi-faceted and covers a broad spectrum of inputs. Itnot only consists of the project binders from the designers, but alsoincludes existing and previous design projects within an organization,plus the linked-in design files, the outputs from the design tools,material from the industry databases (results from contextverification), the configuration scripts, examples, content of sampledatabases provided, legacy databases for known good designs, searchanalytics, information on manufacturing procedures, materialcharacteristics, material prices, and parts that can be obtained fromelsewhere, etc. The result is a sizable database of useable informationavailable to designers and design organizations.

5. An Overall Product Design

FIG. 15 illustrates how a master assembly object (a Level 0 object),along with the requirements that it addresses, can be represented inrelation with sub-assemblies and components (Level 1 objects), alongwith the requirements that the sub-assemblies or components address.FIG. 14 similarly shows how the Level 1 objects can be represented inrelation to corresponding Level 2 objects (together with therequirements that the Level 2 objects address). FIG. 15 further providesan illustration of how the component options and associatedrequirements, for an overall design (one comprising of multiplesubsystems), can be programmed into the database, based on engineeringknowledge gleaned from prior designs. This knowledge may, for example,be related to machine design text awareness of risks for certaincomponents or uses.

6.3 Data Annotation

In order to present a classification system suitable for engineeringproduct design, mission and retail planning, we think of the collectionof archived project binders (e-design notebooks (SteingrimssonYi 2017),(Steingrimsson 2014)) as books in a library. We assimilate the indexingof the project binders to cataloging of books in a library. And wecompare the meta-data defined for the project binders to the indexlabels placed on the books. Similar to the index labels for helping withidentification of books of interest, the meta-data facilities rapidlyprocessing and accurately responding to designer queries. We assume thedesign content gets tagged, in a similar fashion as Google tags allwebsites, to facilitate queries reflecting the users' needs.

6.4 More on Data Tagging: Relations Between Database Objects 1.Relational Databases

Big data analysis capabilities can be applied, for example, atorganizations with design repositories arranged in the form ofrelational databases. The core concept, of project binders accessingdata in databases holding all the design information, and creatingpointers to the pertinent design content, may be adapted to otherdatabase structures.

2. Sample Relations Between Database Objects

FIG. 13 illustrates a sample relation between the customer andrequirement objects. Here we assume, for simplicity, that a givenrequirement originates from a single customer. FIG. 14 provides a sampleillustration of the relation between component or assembly objects andthe requirement objects, respectively. Again, we assume that a componentor assembly object can address many requirements, and that givenrequirement may appear in many components or assemblies.

3. Efficient Tagging Through Index Maps

The relational databases consist of index maps in addition to the data.The index maps allow you to find data efficiently. They mimic tags onbooks in a library. The tagging constructs a map of specific key words,enabling efficient search on a portion of the input design.

4. Note on Data Management

Binders for new design projects are assumed to have the same structureas the binders from the past design projects (and are assumed to bearchived as such). Note that regardless of which Product LifecycleManagement system a design organization elects to use, the design dataneeds to be entered only once. The Ecosystem provides capability forexporting design data into formatted project reports. So the design datadoes not need to be entered more than once. Content from the exportedreports can be used in progress reports or project presentations. Aslong as design organizations make sure that each design project getsarchived after completion, data management is expected to requirerelatively minor effort.

6.6 Querying Engine

In the context of mechanical design, this invention assumes that a userneed occurs as a mix of the following four elements: Function, cost,material, and energy. The first step in the analysis of project binders(e-design notebooks) involves automatic understanding of user needs.Here, the querying engine is expected:

-   1. To comprehend statements of user need/requirement.    -   For this purpose, we treat a user need as a query.-   2. To retrieve previous design information, or examples, that yield    a good match to the user need.

In the process of doing so, the information retrieval frameworkpresented can provide design teams (workforces) with design informationsimilar to the ones previously reported and harvested, for the purposeof enhancing design efficiency and efficacy. To deal with these tasks,we propose to adopt an indexing and retrieval method from the field ofinformation retrieval, one referred to as Latent Semantic Analysis.

1. Latent Semantic Analysis

LSA is an extension of a classic IR model, the Salton's Vector Spacemodel (VSM) (Salton 1975). LSA was developed as an information retrievaltechnique that discovers hidden semantic structure embedded in documents(Deerwester 1990). It is an unsupervised technique for mappinghigh-dimensional count vectors to a lower dimensional representation. Inmore detail, complex relationships exist between words and surroundingcontexts, such as phrases, statements or documents, in which the wordsare located. For the discovery of latent semantic relationships(correlations), LSA begins with the creation of a co-occurrence matrix,where the columns represent contexts and the rows represent words orterms. An entry (i, j) in the matrix corresponds to the weight of theword i appearing in the context j. The matrix is then analyzed byapplying Singular Value Decomposition to derive the associated hiddensemantic structures from the matrix. SVD is a way to factorize arectangular matrix. For an m-by-n matrix, A, with m>n, the singularvalue decomposition of the matrix A is the multiplication of threematrices: An m-by-r matrix U, an r-by-r matrix Σ, and the inverse of ann-by-r matrix V, in that order. That is,

A=UΣV ^(T).  (17)

Here, V^(T) is the matrix transpose of V, obtained by exchanging V'srows and columns. Then, U and V have orthonormal columns and E is adiagonal matrix. Such a multiplication form is referred to as the SVD ofA. The diagonal elements of E are all positive and ordered by decreasingmagnitude. The original matrix, A, can be approximated with a smallermatrix, A_(K), where A_(K) is obtained by keeping the first k largestdiagonal elements of Σ. By definition, k is the rank of the matrix Σ. Byapplying SVD factorization to the matrix A, context (e.g., a set ofstatements characterizing user needs) is represented in a much smallerdimension, k, rather than the original high dimension, m. Note that

k≤n,  (18)

n<<m.  (19)

As a result, a context is represented in a lower dimensional space,rather than in the full, much higher dimension. k is referred to as thedimension of the latent semantic structure of A. A comprehensiveoverview of LSA can be found in (Dumais 2004).

2. LSA-Based Approach

The goal of the LSA-based approach proposed is to provide designers withaccess to previous design records that are relevant to the designers'need. In the vocabulary of Information Retrieving, a designer's need isequivalent to a query.

LSA is adopted as the framework of retrieving designers' needs in thisinvention, because the method has been proven to be an effectiveunsupervised algorithm for IR (Laundauer 2007). In fact, for purpose ofthe querying engine, we are not considering a supervised learningapproach, such as neural networks or deep learning, since such anapproach requires a very large amount of previous e-design examples, orcases, that presently are unavailable.

FIG. 16 depicts an LSA-based approach of retrieving e-design cases thatare likely to satisfy a query (i.e., a user need). The LSA-based methodpredicts the degree of relevance of e-design examples to the query andpresents the most relevant previous e-design cases, or examples, to theuser.

3. Complexity

The algorithms for Latent Semantic Analysis and Latent Semantic Indexingare based on Singular Value Decomposition. The time complexity forextracting the SVD of a m×n matrix is bounded by (Trefethen & Bau III1997)

O(m*n ²)FLOPs.  (20)

Here, m refers to the number of terms, and n to the number of documents.

4. Example Involving Mechanical Product Design

A simple example is presented here to show how the semantic frameworkfor analysis of users' needs can help with idea generation(brainstorming) in the Concept Design stage of a project involving thedesign of a reliable, single-operator Go Kart lift stand. This may be acapstone project, where the experience of the designers in the area maybe somewhat limited. Therefore, they simply pose the following as inputto the querying engine:

“We need a reliable, single-operator stand for kart racers”.

The system responds to the stated need by offering a number of ideas oroptions. Based on what can be retrieved from the databases, or thetraining data available, the system may offer the following suggestions:

-   “1. JEGS Multi-Purpose Lift-   2. KartLift BigFoot-   3. KartLift Winch Lift-   4. Electric Super Lift-   5. Go Kart Stand Lift”.

Supplementing the overall process outlined in FIG. 16, FIG. 18 listsintermediate steps elucidating how the engine for latent semanticanalysis is able to arrive at this conclusion.

While this example may come across as relatively simple (many mechanicaldesigners may have a clue as to what type of lift stands are available),it conveys an application (illustrates the purpose) of the queryingengine. More nuanced examples can be crafted, say, around specificstandards, policies, material properties or common components. To ourknowledge, there is presently no systematic search available for helpingdesigners with brainstorming during concept design.

5. Example Involving Strike Mission Planning

FIG. 20 presents an example showing how the LSA-based approach can beused to retrieve cases that are likely to satisfy an input query from astrike mission planner (i.e., a user need). The LSA-based methodpredicts the degree of relevance of the archived examples to the queryand presents the most relevant previous cases, or examples, to themission planner.

6. Example Involving Surface or Underwater Mission Planning

FIG. 22 presents a similar example showing how the LSA-based approachcan be used to retrieve cases that are likely to satisfy an input queryfrom a surface or underwater mission planner (i.e., a user need). TheLSA-based method predicts the degree of relevance of the archivedexamples to the query and presents the most relevant previous cases, orexamples, to the mission planner.

7. Note about Practicality

Note that the user does not need to worry about parsing of the inputquery. The representation step of the querying engine is invisible tothe user. The user provides the input query in the form of a sentence,such as the ones in FIG. 18, FIG. 20 or FIG. 22. The sentence may evenbe provided verbally with the help of a speech recognition system.

6.7 More on Training for the Predictive Analytics

The predictive analytics framework presented assumes a holistic big dataanalysis and efficient utilization of a broad spectrum of availableinformation. Through proper database representation of the designcontent, in the form of composite design objects (Design[t]), andreferences from the design project journals, one can categorize the dataand run various cross-correlations (queries) across projects or withinprojects. By storing the comprehensive design history in a cloud, andharvesting repositories of known good designs through database queries,one can improve the design decision fidelity for new designs. Access tosuch repositories may also prove invaluable for the purpose ofpost-mortem failure analysis.

The big data frameworks in FIG. 1, FIG. 19, FIG. 21 and FIG. 23 can betrained, for example, using the Delta Rule, mentioned above, which isalso sometimes referred to as the Widrow and Hoff learning rule, or theLeast Mean Square rule (Widrow 1960):

$\begin{matrix}{{\Delta \; w_{ijx}} = {{{- ɛ}\frac{\delta \; E}{\delta \; w_{ij}}} = {ɛ\mspace{14mu} \delta \; {a_{ix}.}}}} & (16)\end{matrix}$

Here Δw_(ijx) represents the update applied to the weight at node(perceptron) between links i and j in a neural network (Widrow 1960). Erepresents an error function over an entire set of training patterns(i.e., over one iteration, or epoch) (Widrow 1960). ε is a learning rateapplied to this gradient descent learning. a_(ix) denotes actualactivation for node x in output layer i (Widrow 1960).

6.8. Validating the Integrity of Archived Data—The Impact of IncompleteData 1. Predictive Analytics

We put high emphasis of only training the system model on known gooddesign (designs qualified as having all requirements fulfilled), eventhough

-   1. It is possible you get a guiding reference of some value, even if    you relax the constraint about the archived designs being qualified    as good;-   2. It is of course much easier to get started if you relax the    constraint, esp. for organizations with large databases of legacy    designs.-   3. We are recommending the neural network solution in the event of a    large set of archived binders.    -   In the event of a small to intermediate number of archived        binders, we are recommending multivariate linear regression for        training the system model.

Once you relax the constraint of the archived designs being of goodquality, the quality of the output will be un-deterministic. In theworst case, the quality of the guiding reference may soon be impacted(significantly?), if one were to relax the constraint on the baddesigns. Are 90% of the designs good or only 10%? Even if the systemmodel were trained based on designs, only 50% of which had beenqualified as good, the quality of the guiding reference might besuspect. We want to take steps to protect against the possibility of“garbage in” producing “garbage out”.

2. Adopting Designs from a Legacy Database (Whose Designs are Yet to beQualified as Good)

As a practical way to adoption of the big data framework by companies,that have a large database of legacy designs, that are yet to bequalified as good, we recommend incremental re-training.

Even in the case of a large database of legacy designs, we recommendstarting out by identifying a (small) set of good designs on which totrain the system model. Then you incrementally expand the training set,but qualify the new designs from the legacy database, either throughautomatic requirement verification provided by SW like the Ecosystem orby a human. Then you retrain the system model, once you have qualifiedthe new input. In this way, you can expand the training set in acontrolled fashion, and refine the system model, and yet maintainquality of the archived designs comprising the training set.

2. Querying Engine

Most search engines are based on similar concepts as the querying enginepresented (information retrieval). LSA is less sensitive to datascarcity than many supervised machine learning techniques. It's assumedthat the LSA returns hits closely matching an input query. With anappropriate threshold being set, LSA can tell how many matches there areto the input query. In this sense, the querying engine is capable ofgracefully accounting for cases of incomplete data.

6.9 Accommodations Specific to Incremental or Remedial DesignProjects 1. Single Reference Design

In case of an incremental or remedial design project, based on a singlereference design, there is no need for predictive analytics. As opposedto generating a guiding reference, y, through predictive analytics, theoriginal reference is the guiding reference.

2. (Small) Sub-Set of Reference Designs

The matrix {tilde over (W)} enables specification of the (small) sub-setof designs to be used reference in this case. Here, one could stillproduce the guiding reference as

y={circumflex over (B)} ^(T) x.

But {circumflex over (B)} would be obtained as solution to (14), whererow j of Ŵ consisted of all ones if and only if archived vector{circumflex over (x)}_(j) was among the few vectors in the (small)sub-set.

6.10 Accommodations Specific to Mission Planning 1. Accommodations ofHighly Dynamic Environments—Dynamic Re-Planning

Dynamic re-planning can be accounted for relatively easily, inprinciple: Once you have a new input (a new “requirement” vector), youreceive the system output (the guiding mission plan) nearinstantaneously.

If things get heated in battle, due to an action of an adversary, anddecisions needed to be made on the fly, we may hold off presenting thecorresponding guiding advisories, unless components of the guiding planfulfilled minimum quality criteria.

Further, as new information become available, one can conductincremental training updates of the system model. The framework forpredictive analytics employs supervised learning. The system is trainedonce in the beginning based on the archived

-   -   (Requirement, Mission plan)        duplets available. But the longer the trained system is used,        and as more information becomes available, one can conduct        incremental training updates.        2. Rapid and Continuous Planning when New Information        Necessitates Updating

Similarly, rapid and continuous planning can be accounted for relativelyeasily, in principle: Once you have a new input (a new Requirementvector), you receive the system output (the guiding Mission Plan) nearinstantaneously.

3. Multi-Vehicle, Multi-Domain Mission Planning: Generation of MissionPlans in a Near-Autonomous Mode, Given the Current Workflow for MissionPlanning

Key to applying the framework for predictive analytics to multi-vehicle,multi-domain mission planning is to “correctly” collect data into thedatabase and tag. All the vehicles and domains supported can have auniversal index, and the indices can be recorded as a part of the“requirement” objects, as shown in Table 11. There may be a “local”version of the database, for each vehicle and/or domain. As you collectdata into the “local” versions, you may tag the data to indicate whichdomain or asset it corresponds to. Then you may have a “global”,centralized database for all the assets and domains, with whom the“local” versions synchronize their data.

4. Accounting for Different Security Levels

In combat, the crew that plans a mission is typically not the crew thatexecutes it, so decisions are sometimes made to enable as muchflexibility as possible, and delegate the decision authority to thelowest possible level (i.e. the person in the cockpit). Our generalapproach will be to comply with Navy protocols in this regard. As shownin Table 11, the security level can be recorded as a tag in the databasefor the input “requirements”, in the archived duplets for

-   -   (Requirement, Mission plan).

One can record the security levels using a standardized scheme.

6.10. Support for Pertinent Interfaces

1. Interface with Spark or Hadoop for Big Data Analysis

Regarding the type of interfaces that the system will be able to support(regarding the system's ability to support both heterogeneous(distributed) and homogeneous architectures), one should note thatlarge, distributed databases (enterprise applications), such as ApacheHadoop or Spark, can be supported through the API interfaces provided bythese tools (Spark 2017). Hadoop provides a native Java API to supportfile system operations (BigData 2017). One can use WebHFDS to interactwith the Apache Hadoop file system externally through a more userfriendly REST API (Hadoop 2017). WebHDFS concept is based on HTTPoperations like GET, PUT, POST and DELETE (Hadoop 2017). Operations likeOPEN, GETFILESTATUS, LISTSTATUS are using HTTP GET, while others likeCREATE, MKDIRS, RENAME, SETPERMISSIONS are relying on HTTP PUT (Hadoop2017). The APPEND operation is based on HTTP POST, whereas DELETE isusing HTTP DELETE (Hadoop 2017). Authentication can be based onuser.name query parameter (as a part of a HTTP query string). Ifsecurity has been turned on, then the authentication relies on Kerberos(Kerberos 2017).

2. Interface with JMPS System for Strike Mission Planning

The Joint Mission Planning System is a software application thatconsists of a basic framework together with unique mission planningenvironment software packages for various platforms (TianWeiYaoping2016). FIG. 24 outlines one way of integrating the big data frameworkinto JMPS. We are presenting a high-level framework of learning methodsthat can overlook JMPS. The big data framework can offer advice to JMPS,based on outcomes and performance data provided.

3. Interface with the MEDAL System for Surface and Underwater MissionPlanning

FIG. 25 shows how the big data framework can offer advice to theShipboard MEDAL, the CMWC MEDAL and the data warehouse systems, based onoutcomes and performance data provided. In current mine warfare missionplanning, historical oceanographic data are acquired from the NAVOCEANOdatabase, also listed in FIG. 25. These data, including seafloorcharacteristics, water column properties, and atmospheric parameters,are input into MEDAL to determine optimal lane spacing and predictoperational time lines and risk. Once into the exercise, in situ dataare collected and input into MEDAL and other tactical decision aids,used for naval warfare mission planning. When in situ data are enteredinto MEDAL, the operator can output predictions of best and worst-casescenarios for line spacing to balance operational objectives withclearance time and risk. It is important at this stage of operationsthat mine warfare personnel be receptive to modifying tactics to fitchanges in environmental conditions.

7. How to Use the Invention

1. For Improving Fidelity of Design Decisions Through Comparison with aReference Forecast

FIG. 1 summarizes this application. The vector y captures the referenceforecast. When decisions made during the course of a new design projectdeviate from the reference, this prompts a case (anomaly) likely worthinvestigating. The invention addresses harvesting of information frompast engineering design projects for the purpose of aiding future designprojects.

2. For Idea Generation: Querying for Solution Options

FIG. 17 and FIG. 18 summarize such an application.

3. For Querying for Common Components

This invention can be used to query external databases for commoncomponents used in a design, as FIG. 1 highlights.

4. For Querying for Information Related to Standards

This invention can be used to query external databases for engineeringstandards pertinent to a design, as FIG. 1 highlights.

5. For Querying for Information Related to Material Properties

This invention can be used to query external databases for theproperties of materials considered for us in a design, as FIG. 1highlights.

6. For Querying for Information Related to Regulations

This invention can be used to query external databases for regulationsthat may impact a design.

7. For Querying for Information Related to Policies

This invention can be used to query external databases for policies thatmay impact a design.

8. For Querying for Customer Related Information

This invention may be used to query for information related to customersinvolved in a design.

9. For Querying for Information Related to Internal Requirements

This invention may be used to query for information related to internalrequirements involved in a design.

10. For Querying for Information Related to Best Practices

This invention may be used to query for information related to bestpractices involving a design.

11. For Querying for Information Related to Previous Solutions

This invention may be used to query for information related to previousdesign solutions.

12. For Querying for Information Related to Analogies

This invention may be used to query for information related to analogoussolutions.

13. For Querying for Other Information Related to Detailed or FinalDesign

This invention may be used to query for other information related to thedetailed or final design stages of a design project.

14. For Design Projects Involving Incremental Changes or RemedialEfforts

This invention can be used on design projects involving incrementalchanges or remedial efforts, as explained in 6.9 above. The driver mayinvolve cost savings, material change, etc.

15. For Design to Manufacture, or Simply for Manufacture

The predictive analytics and the querying engine may be used both todesign an engineering product as well as to design the equipment used tomanufacture the design. In case of mechanical design in industry,separate department may be responsible for designing parts and designingthe molds that make the parts. Decisions made during the design of themolds can have significant cost implications. For example, inmanufacture of a key board, do you manufacture the keys in a group orindividually? Are you going to use single-shot molding or double-shot?If you misplace the position of the screw, where the plastic flows intothe mold, there can be significant cost penalty.

16. For Maritime Mine Detection and Neutralization (for Surface orUnderwater Mission Planning)

This invention may be used for continuous planning for maritime minedetection and neutralization using unmanned vehicles. Assets involvedmay include submarines or ships to be protected from the mines. Theocean environment significantly influences mine warfare tacticalplanning (OceanStudies 2000). Understanding of nearshore environmentalvariability is important, not only for physical mine placement but alsofor its impact on mine hunting sensors (OceanStudies 2000). The coastalenvironment tends to be complex and may lack of high-resolutionnearshore oceanographic and bathymetric data, particularly in areas ofpotential conflicts (OceanStudies 2000).

17. For Guiding Near-Autonomous Generation of Strike Mission Plans

For specifics on application of the invention to strike missionplanning, refer to FIG. 19 and FIG. 20.

18. For Automatically Acquiring and Continually Updating AssetPerformance Models and Tactical Planning Knowledge, in Order to ImproveDecision Support by Automated Mission Planning Systems

For specifics on application of the invention to surface or underwatermission planning, refer to FIG. 21 and FIG. 22 FIG. 25 shows how theinvention can fit into the information systems presently used, such asthe Mine Warfare Environmental Decision Aid Library.

19. For Use in Conjunction with Automatic Requirement Verification

The invention can be used in conjunction with automatic requirementverification, such as within the automotive or avionics industries. Theinvention can help contribute to the making of safe autonomous vehicles,vessels or aircrafts.

20. For Mission Command

The invention can harvest operational data for the purpose of providingpredictions, alerts, and recommendations. An autonomous learned system(machine learning) can understand large amounts of data, manage theresults, and react to cyber defense, electronic warfare, and even largeraid attacks. The big data framework can help enhance human performancein the processing of information management and knowledge managementduring exercise of Mission Command.

21. For Retail Planning and Supply Chain Management

For specifics on application of the invention to retail planning andsupply chain management, refer to FIG. 23.

22. Other Applications

Other private sector applications may include survey and first responderoperations. Solutions for multi-vehicle, multi-domain mission planningmay benefit various companies that deal with parcel delivery such asAmazon, UPS, FedEx, and others by generating autonomous mission plans(optimized delivery plans for either multiple ground and air vehicles).Another field that may benefit is traffic engineering, providing andynamic approach to traffic control based on various traffic conditions.

8. Further Examples of the Invention

Thus, it will be appreciated by those skilled in the art that thepresent invention is not restricted to the particular preferredembodiments described with reference to the drawings, and thatvariations may be made therein without departing from the scope of theinvention

This invention claims:
 1. An apparatus for predictive analytics, onethat utilizes vector entities archived from past design or planningprojects, for the purpose of providing predictions aiding new design orplanning projects, an apparatus comprising of a database module forstoring the vector entities, with one vector entity representing aninput vector from a past design or planning project, and the anothervector representing a corresponding known good output vector, with aknown good output vector being defined as an output vector that fulfillsoriginal requirements imposed on the archived design or planningproject, a generic system module, for relating the output vector to theinput vector, whose system model is derived from the archived vectorentities, and where the system module can be made capable of supportinginput vectors with elements taking on continuous or categorical values,where the values are confined to a range or not, the system module canbe derived using multivariate linear regression, especially in case ofinput data set of small to medium size, or can be implemented usingneural networks, especially in case of input data set of large size, thesystem module can be made capable of approximating a genericcontinuous-valued system model with a two-layer feed-forward neuralnetwork, a prediction module for producing a guiding reference for a newdesign or planning project, where the guiding reference is obtained asoutput from the system module when applying an input vector from the newdesign or planning project as input, an optional external databasemodule, for querying for pertinent reference information helpful to thecontext of the new design or planning project, when necessary.
 2. Amethod for predictive analytics, one that utilizes vector entitiesarchived from past design or planning projects, for the purpose ofproviding predictions aiding new design or planning projects, a methodthat further utilizes a database access step, for accessing the vectorentities, with one vector entity representing an input vector from apast design or planning project, and another vector representing acorresponding known good output vector, with a known good output vectorbeing defined as an output vector that fulfills original requirementsimposed on the archived design or planning project, a generic systemmodel access step, for relating the output vector to the input vector,where the system model is derived from the archived vector entities, andwhere the system model can be made capable of supporting input vectorswith elements taking on continuous or categorical values, where thevalues are confined to a continuous range or not, the system model canbe derived using multivariate linear regression, especially in case ofinput data set of small to medium size, or can be implemented usingneural networks, especially in case of input data set of large size, thesystem model can be made capable of approximating a genericcontinuous-valued system model with a two-layer feed-forward neuralnetwork, a prediction step, for producing a guiding reference for a newdesign or planning project, a reference obtained as output from thesystem model when applying a vector from the new design or planningproject as input, an optional access step to an external database ordatabases, for querying for pertinent reference information helpful tothe context of the new design or planning project, when necessary.
 3. Anapparatus according to claim 1 for improving the fidelity of engineeringdesign decisions on new design projects, an apparatus that utilizesvector entities from past design projects, the composition of saidvector entities includes an input vector, which captures product designspecification, an optional customer object, which is a part of the inputvector, a requirement object, which is also a part of the input vector,an output vector, which captures design data, an optional componentobject, which is part of the output vector, and an optional assemblyobject, which is also a part of the output vector.
 4. A method accordingto claim 2, for improving the fidelity of engineering design decisionson new design projects, a method utilizing vector entities from pastdesign projects, the composition of said vector entities includes aninput vector, which captures product design specification, an optionalcustomer object, which is a part of the input vector, a requirementobject, which is also a part of the input vector, an output vector,which captures design data, an optional component object, which is partof the output vector, and an optional assembly object, which is also apart of the output vector.
 5. A method according to claim 4, where thestructure of the database objects is determined based on function, andwhere the definition of a customer object includes a specification of aunique identifier for the customer, a specification of a name for thecustomer, an optional specification of a category for the customer, anoptional specification of an importance level assigned to the customer,a method capable of interfacing with Hadoop and Apache Spark softwareframeworks, for distributed processing of large data sets, throughapplication program interfaces provided by these software frameworks. 6.A method according to claim 4, where the structure of the databaseobjects is determined based on function, and where the definition of arequirement object includes a specification of a unique identifier forthe requirement, a specification of a name for the requirement, anoptional specification of a customer with whom the requirement isassociated, an optional specification of an importance level assigned tothe requirement, an optional specifications of a category for therequirement, an optional specification of a function that therequirement is addressing, an optional specification of characteristicsfor the requirement, an optional specification of the minima and maximadefining the range, or ranges, of values corresponding to therequirement being fulfilled, an optional specification of the unitassociated with the range, or ranges, of values corresponding to therequirement being fulfilled.
 7. A method according to claim 4, where thestructure of the database objects is determined based on function, andwhere the definition of an assembly object includes a specification of aunique identifier for the assembly object, a specification of a name forthe assembly object, a specification of the requirement, orrequirements, that the assembly object addresses, an optionalspecification of an assembly, or assemblies, subordinate to the assemblywhich the assembly object describes, an optional specification of theinput applied to the assembly which the assembly object describes, anoptional specification of the output produced by the assembly which theassembly object describes, an optional specification of the processapplied to the assembly which the assembly object describes, an optionalspecification of the hierarchical rules or constraints, that theassembly, which the object describes, is subjected to.
 8. A methodaccording to claim 4, where the structure of the database objects isdetermined based on function, and where the definition of a componentobject includes a specification of a unique identifier for the componentobject, a specification of a name for the component object, aspecification of the requirement, or requirements, that the componentobject addresses, an optional specification of the input applied to thecomponent, which the object describes, an optional specification of theoutput produced by the component, which the component object describes,an optional specification of the process applied to the component, whichthe component object describes, a specification of the dimensions thatapply to the component object, an optional specification of the materialcharacteristics for the component object.
 9. A method according to claim2, for identifying anomalies in a set of continuous-valued orcategorical data, a method employing comparison of distribution of eachdata variable, either by using quantiles or another statistical test, tosee if the variations are significantly different, or counting ofoccurrence of each label or category and comparing, or a distancemeasure, such as the Mahalanobis distance, and looking for big changes,or applying an absolute difference between new and old data, setting athreshold, and reporting everything exceeding the threshold, or amulti-dimensional technique, like correlation matrix, principalcomponents or clustering, and looking for changes, or a statistical ormachine learning model, specialized for anomaly detection, such asSupport Vector Machines, t-distributed Stochastic Neighbor Embedding,Isolation Forests, Peer Group Analysis or Break Point Analysis.
 10. Amethod according to claim 4, where a mechanical engineering design isrepresented in a database in the form of a hierarchical structureconsisting of a master assembly object, along with the requirements thatthe master level object addresses, sub-assembly objects and components,that depend on the master assembly object, along with the requirementsthat the sub-assembly or component objects address, sub-sub-assemblyobjects and components, that depend on the sub-assembly objects, alongwith the requirements that these sub-sub-assembly or component objectaddress, subsequent levels of hierarchical representation of assemblyobjects and components, together with the requirements that theseobjects address, as needed to represent the design with the accuracy anddetail desired.
 11. A method according to claim 4, where the database isstructured such that component options and risks can be programmed intothe database based on existing engineering knowledge.
 12. An apparatusaccording to claim 1, for automatically acquiring and continuallyupdating asset performance models or tactical planning knowledge, inorder to improve decision support by automating mission planning inhighly dynamic environments, and for enabling maintainability, anapparatus comprising of an input vector capturing requirements for atactical mission plan, an object, which is a part of the input vector,for specifying mission goals, an optional object, which is also a partof the input vector, for specifying expected mission performance, anoutput vector consisting of a tactical mission plan or an asset model,an optional object, which is part of the output vector, for registeringa performance capability, and an optional object, which is part of theoutput vector, for registering an environmental model, an apparatuscapable of interfacing with a Mine Warfare Environmental Decision AidLibrary, a data warehouse, or another Tactical Decision Aid used fornaval warfare mission planning.
 13. A method according to claim 2, forautomatically acquiring and continually updating asset performancemodels and tactical planning knowledge, in order to improve decisionsupport by automated mission planning systems in highly dynamicenvironments, and for enabling maintainability, a method utilizing aninput vector capturing requirements for a mission plan, an object, whichis a part of the input vector, for specifying the mission goals, anoptional object, which is also a part of the input vector, forspecifying the expected mission performance, an output vector consistingof a tactical mission plan or asset model, an optional object, which ispart of the output vector, for registering the performance capability,and an optional object, which is part of the output vector, forregistering the environmental model, a method capable of interfacingwith a Mine Warfare Environmental Decision Aid Library, a datawarehouse, or another Tactical Decision Aid used for naval warfaremission planning.
 14. An apparatus according to claim 1, for guidingnear-autonomous generation of strike mission plans, an apparatusleveraging data, models and standards in a database for missionplanning, an apparatus capable of supporting multi-domain, multi-assetmission planning as well as dynamic re-planning, an apparatus comprisingof an input vector, for capturing requirement definition for a missionplan, an object, which is a part of the input vector, for capturing themission goals, an optional object, which is also a part of the inputvector, for capturing waypoints on the mission route, an output vectorconsisting of the mission plan itself, an optional object, which is partof the output vector, for capturing weapon selected, an optional object,which is part of the output vector, for capturing waypoint selected, anoptional object, which is part of the output vector, for capturingassets coordinated, an optional object, which is part of the outputvector, for capturing threats assessed, an optional object, which ispart of the output vector, for capturing strike composition, an optionalobject, which is part of the output vector, for capturing fuel usage, anoptional object, which is part of the output vector, for capturing timeline developed, and an optional object, which is part of the outputvector, for capturing communication plan, an apparatus capable ofinterfacing with a Joint Mission Planning System for strike missionplanning.
 15. A method according to claim 2, for guiding near-autonomousgeneration of strike mission plans, a method leveraging data, models andstandards in a database for mission planning, a method utilizing aninput vector containing a requirement definition for a mission plan, anobject stating the mission goals, which is a part of the input vector,an optional waypoint object, which is also a part of the input vector,an output vector consisting of the mission plan itself, an optionalobject, which is part of the output vector, capturing weapon selected,an optional object, which is part of the output vector, capturingwaypoint selected, an optional object, which is part of the outputvector, capturing assets coordinated, an optional object, which is partof the output vector, capturing threats assessed, an optional object,which is part of the output vector, capturing strike composition, anoptional object, which is part of the output vector, capturing fuelusage, an optional object, which is part of the output vector, capturingtime line developed, and an optional object, which is part of the outputvector, capturing communication plan, a method capable of interfacingwith a Joint Mission Planning System for strike mission planning.
 16. Anapparatus according to claim 1, for improving retail planning and supplychain management, by validating forecasted category sales and sanitizingthe planning process through provision of guiding references, anapparatus comprising of an input vector containing forecasted sales bystore and category, an output vector consisting of the observed sales bystore and category.
 17. A method according to claim 2, for improvingretail planning and supply chain management, by validating forecastedcategory sales and sanitizing the planning process through provision ofguiding references, a method utilizing an input vector containingforecasted sales by store and category, an output vector consisting ofthe observed sales by store and category.
 18. A method according toclaim 2 for integration of digital product metrology informationobtained from a large database using predictive analytics throughmetrology feature recognition, and for generation of an inspection planfrom the recognized features.
 19. A method for querying a database, forthe purpose of efficiently identifying archived database items matchinga new user input query, a method utilizing a representation step,involving a sequence of text processes, for purpose of identifyingvocabulary of terms in the user input query or in corpus of databaseitems, a weighting step, in which a weighting scheme is applied to thevocabulary of terms, with resultant weights becoming values in a matrixfor latent semantic analysis, a comparison step, based on latentsemantic analysis, in which the degree of relevance between the inputquery and the corpus of database items is predicted, for the purpose ofefficiently matching the input query with the corpus items, a retrievalstep, for retrieving the database corpus items most relevant to the userinput query.
 20. A method according to claim 19, in which therepresentation step consists of tokenization, dropping common terms andstop words, normalization of terms, and stemming.
 21. A method accordingto claim 19, in which the semantic analysis in the comparison stepinvolves creation of a co-occurrence matrix, where the columns representcontext or documents and the rows represent words or terms, a methodwhere an entry (i,j) in the matrix corresponds to weight of word iappearing in context j, a method in which singular value decompositionis applied to the co-occurrence matrix, for the purpose of identifyinghidden semantic relationships.
 22. A method according to claim 2, forguiding near-autonomous generation of strike mission plans, a methodleveraging data, models and standards in a database for missionplanning, a method capable of supporting multi-vehicle, multi-domainmission planning in highly dynamic environments along with appropriatesecurity levels, a method utilizing for this purpose requirement objectsfeaturing a domain parameter, capturing a universal index to the missiondomains, an asset parameter, capturing a universal index to the missionassets, a parameter accounting for the different security levelssupported, capturing a universal index to the mission assets, anoptional parameter specifying mission target selected, an optionalparameter specifying location of the mission target selected, anoptional parameter specifying the overall mission goals.